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[ Text ] 1 . Introduction 

For the image processing of an object in its natural image, it is necessary 
to extract in advance the object to be processed from its image. To 
accomplish this the outer shape of an object is extracted through human 
instructions, which requires a great deal of time and patience. 

This paper describes a method involving the setting of a model of color 
distribution on the surface of an object, thereby automatically providing 
"color" recognition, a piece of knowledge to represent the properties of an 
object, from its natural image. In addition, it describes a method for 
recognizing and extracting the object in the image according to the color 
recognized. 

2. Setting of Surface Model of Reflected Objects 

In computer graphics, various methods of expressing the surface of an 
object have been proposed in order to obtain realistic images. The authors 
studied a color distribution model with the light constantly contained in 
natural images taken into consideration by applying a model phong, which 
is often used for expressing bright surfaces, etc. 

A phong model is often used for plastic-like material expression. The 
reflected light S p at point P on the surface can be expressed by the 
expression (1): 

S p (A) - C (A) { cos ( i ) (1 - d) + d) 

+ W(A,i) ( cos ( s ) } n (1) 
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where 

C p : diffuse reflectance at point P on an object 

i: angle of incidence of illumination light 

w: mirror reflectivity 

s: angle between the directions of mirror reflection and the line of 

sight 

n: power value for mirror reflection model by each material 

d: ambient light reflection coefficient 

Since S p is a function of light wavelength A, the three stimulus values of 
R, G, and B can be obtained by integrating them with sensor sensitivity; 

/ R\ r r(A) 

G - / S p (A) g(A) dA 

\ B / b(A) (2) 

Since the terms dependent on wavelengths are C (A) and W(A,i) alone, 
substituting expression (1) for expression (2), we have 


where 



R o\ 

G 0 {cos(i) (1 - d) + d) 



W(A, i) {cos(s) } n 


R o - S C p (A)r(A)d A 
G 0 - / C p (A)g(A)d A 
B o - / C p (A)b(A)d A 


R g W(i) - f W(A , i)r(A)d A 
G 8 W(i) - / W(A , i)g(A)d A 
B s W(i) - / W(A , i)b(A)d A 


(3) 


It is considered that W(i) is dependent on color term i of the light source 
and is separable. Setting a - cos(i) <1 - d) + d, f) - W(i) {cos(s)} n , we 
find that expression (3) results in 



o' ''"S' 

(0 £ a £ 1, 0 £ p £1), 


where (R 0 , G 0 , B 0 )t and (R a , G s , B a )t can be regarded as colors appearing 
in diffuse reflection of an object (hereafter called object color) and 
those of illumination light (hereafter called light source color) , 
respectively. It is considered that the color of reflected light at each 
point changes according to the surface direction and illumination at each 
point and the positional relation between itself and the observer, and that 
these factors are mixed at rates of a and p . 

If the expression (4) model is correct, the color of the object surface 
must exist in a plane passing the black, the light source color and the 
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object color. The report by Taj ima and his colleague 1 indicates that the 
color of an object is distributed on a plane and that the slippage between 
this plane and the one passing the black, the light source color and the 
object color is about 5 degrees, with these two planes almost according 
with each other. 

Figure 1 shows a projection drawing in which the body color of an 
automobile parked on a street is projected on this distribution plane with 
the black-white axis from the black (0,0,0) in RGB space to the white 
(255,255,255) displayed at the same time. From the picture, the light 
source color seems to be white, while the color of the object surface is 
distributed not only in the plane decided by the black, light source color 
and object color, but also in a triangle with the black, light source color 
(white) and object color as its vertexes. From this, expression (5) is 
considered to hold: 


R \ / R„ \ / R. \ 

0 £ a £ 1, 0£/3£l, 




(5) 


We will call this color distribution model a reflected object surface 
model . 



Figure 1. Color Distribution in Automobile Body 
3. Verification of Reflected Object Surface Model 

We decided to verify the efficiency of the reflected object surface model 
of the expression (5) proposed in section 2 with several natural images. 
The processing for the distribution drawing in Figure 1, however, was 
conducted by deciding a distribution plane after the dispersion was 
obtained, so that it required about 40 minutes with a minicomputer with 
processing performance of 3 mips. To facilitate the verification, we 
considered a method for expressing color distribution without seeking 
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Figure 2. Color Distribution in Automobile Body 

dispersion and succeeded in the processing above with a 16-bit personal 
computer. Figure 2(a) and (b) show the result of the above processing for 
the same picture of the automobile. 

The axis of abscissas in Figure 2(a) is the black-white axis where factors 
of R, G, and B become equivalent, while its axis of ordinates represents 
the distance from the black-white axis. Figure 2(b) shows individual 
points projected on plane vertical to the black-white axis. If examination 
of a natural image of an object other than an automobile shows its color to 
be distributed as in this distribution drawing, it will demonstrate that 
the reflected object surface model is correct. 



Figure 3. Color Distribution in Cleaner 

Figure 3(a) and (b) show a color distribution of a natural image of a 
cleaner with a plastic-like surface for a phong model. Here, too, the 
light source color can be assumed to be white, and it can be found that as 
is the case with an automobile, the color is distributed in a triangle with 
the black, light source color and object color as its vertexes. In 
addition, the authors examined such objects of materials with great mirror 
reflection as a TV set, a telephone and a desk, and obtained a similar 
result, thus confirming the reflected object surface model as correct. 



Figure 4. Color Distribution in Suit 
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With respect to the surface of a suit material lacking mirror reflection, a 
similar result to that shown in Figure 4(a) and (b) was obtained, 
indicating that its color is distributed in the same way. With respect to 
interior light equipment, which has great mirror reflection and is made of 
material permitting light to permeate, the result shown in Figure 5(a) and 
(b) was obtained, and it has been confirmed that its color is distributed 
as well. 



Figure 5. Color Distribution in Interior Lighting Equipment 

From the above results, it has been confirmed that the color of the surface 
of an object, irrespective of the materials used in the object surface, is 
distributed in a triangle, with the black, light source color and object 
color as its vertexes. Therefore, the reflected object surface model the 
authors have proposed is applicable, irrespective of the materials used in 
the object surface. 

4. Recognition of Object Color and Extraction of Object Domain 

Since the light source color can very often be assumed as white, as is the 
case with sunlight the light source color is regarded as white in the 
following description. In a reflected object surface model, the color of 
the object surface exists on a plane with a black- white axis as its bottom 
end and the object color itself is the one farthest from the black-white 
axis. Therefore, if any color on an object is specified and its 
distribution plane is decided, the color farthest from the black-white axis 
on this plane can be recognized as the object color. The authors 
considered that by doing so, what entered the scope of a triangle with the 
white, black, and object color as its vertexes could be automatically 
extracted as a portion of the object. 

However, as verified in section 3, the object color in a natural image is 
distributed not in a perfect plane, but in a certain width. The lightness 
of color changes due to the effect caused by light sources such as shadows, 
but the hue does not, so it can be considered that the width of color 
distribution extends in the direction vertical to the hue surface. When 
viewed in the color distribution in Figure 2, since the plane with the 
black-white axis as its end can be approximately regarded as the hue 
surface in RGB space, the water droplet type shown in Figure 2(b) can be 
regarded as the width of color distribution. For practical performance, 
the authors conducted an experiment having a water droplet type approximate 
to a fan type, confirming that use of a fan- type model with a given 
interior angle permits the object color to be recognized without posing any 
problem for its practical use. 
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Table 1. Width of Color Distribution 


Object 

Interior angle 

Automobile 

58 

Cleaner 

42 

Suit 

37 

Interior lighting equipment 

28 


The object color could be recognized by using a fan- type model with a given 
interior angle; however, with respect to the automatic extraction of a 
portion of an object, since the width of the interior angle of one image is 
largely different from that of another, the unrecognizable portion of the 
object increases when the interior angle is too narrow, while something 
other than the object is also recognized by mistake when the width of the 
interior angle is too wide. For the automatic extraction of object domain, 
it is necessary to strictly set the interior angle. The authors succeeded 
in extracting object domain by providing the dispersion of hue in terms of 
parameter through visual observation. 

5. Conclusion 

The authors have so far conducted an experiment on the automatic 
recognition of object domain from its natural image, but they could not 
automatically recognize the width of color distribution. From the analysis 
result of natural images obtained so far, the width of color distribution 
is considered to be caused by materials of the object surface; therefore, 
if knowledge of materials with the width of color distribution taken into 
consideration can be obtained, more perfect automatic extraction of object 
domain together with color will be possible. 
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